Signals carried over telecommunications links can undergo considerable transformation, such as digitisation, data compression, data reduction, amplification, and so on. Further distortions can be caused by electromagnetic interference from external sources.
Objective processes for the purpose of measuring the quality of a signal are currently under development and are of application in prototype testing, pre-delivery testing of components, and in-service testing of installed equipment. They are most commonly used in telephony, but are also of application in other systems used for carrying speech signals, for example public-address systems.
The present applicant has a number of patents and applications relating to this technical field, most particularly European Patent 0647375, granted on 14 Oct. 1998, and its foreign equivalents. In this invention, two initially identical copies of a test signal are used. A first copy is transmitted over the communications system under test. The resulting signal, which may have been degraded by the system under test, is compared with a reference copy of the same signal, which has not passed through the system under test, to identify audible errors in the degraded signal. These audible errors are assessed to determine their perceptual significance that is, errors of types which are considered significant by human listeners are given greater weight than are those which are not considered so significant. In particular, inaudible errors, being perceptually irrelevant and therefore unnecessary for assessment, are disregarded.
The automated system provides an output comparable to subjective quality measures originally devised for use with human subjects. More specifically, it generates two values, YLE and YLQ, equivalent to the “Mean Opinion Scores” (MOS) for “listening effort” and “listening quality”, which would be given by a panel of human listeners when listening to the same signal. The use of an automated system allows for more consistent assessment than human assessors could achieve, and also allows the use of compressed and simplified test sequences, which give spurious results when used with human assessors because such sequences do not convey intelligible content.
Different errors may have different perceptual significance in different languages, according to the use of different phonemes, and their relative frequencies of occurrence, in different languages. Multilingual test sequences are also more readily accommodated in automated systems than those using human subjects, as the human subjects would need to be familiar with all the languages represented in the test signal, again to avoid spurious results caused by any lack of intelligibility to the listener.
An auditory transform of each signal is taken, to emulate the response of the human auditory system (ear and brain) to sound. The degraded signal is then compared with the reference signal in the perceptual domain, in which the subjective quality that would be perceived by a listener using the network is determined from parameters extracted from the transforms.
Such automated systems require a known (reference) signal to be played through a distorting system (the telephone network) to derive a degraded signal, which is compared with an undistorted version of the reference signal. Such systems are known as “intrusive” measurement systems, because whilst the test is carried out the system under test cannot carry live (revenue-earning) traffic.
A suitable test signal is disclosed in European Patent Specification 0705501 and comprises a sequence of speech-like sounds, selected to be representative of the different types of phonetic sounds that the system under test may have to handle, presented in a predetermined sequence. The sounds are selected such that typical transitions between individual phonetic elements are represented. Typical speech comprises a sequence of utterances separated by silent periods, as the speaker pauses to breathe, or listens to the other party to the conversation. These silent periods, and the transitions between utterances and silent periods, are also modelled by the test signal.
The arrangement described above requires the use of a pre-arranged test sequence, which means it cannot be used on a live telecommunications link—that is, a link currently in use for revenue-earning traffic—because the test sequence would interfere with the traffic being carried and be audible to the users, and because conversely the live traffic itself (whose content cannot be predetermined) would be detected by the test equipment as distortion of the test signal.
In order to carry out tests on equipment in use, without interfering with the revenue-earning traffic being carried by the equipment (so-called non-intrusive testing), proposals have been made to use the live speech signals themselves as the test signals. This can be achieved quite simply if an undegraded copy of the original signal is available at the point of measurement. In test conditions it is, of course, possible to place the transmitter and receiver in close proximity, but this is impossible when testing an in-service long-distance link. In a real situation any reference signal is likely to be subject to similar degradation to that experienced by the link under test, unless a significant error correction overhead is provided to ensure that the reference copy is itself received undegraded.
German Patent Specification 4324292 (DeTeCon) of 1995 discloses a system in which an empirical relationship is determined between an objective measure (such as bit-error rate) and a subjective measure (that is, one made by human subjects), such as Mean Opinion Score. This derived relationship is then used to estimate the subjective quality of a received signal from an objective measure made of the live network. For this arrangement to produce reliable results sufficient repetitions of each subjective test must be made to provide a statistically reliable value for the Mean Opinion Score. Such a system is not practical for any but the simplest of systems. The search space for determining an accurate relationship between an objective measure such as packet statistics and a subjective measure of speech quality is very large. To determine a person's opinion of speech quality needs several seconds of speech to analyse. However, it can take many days or even weeks to analyse the objective speech properties and assess them relative to the subjective values reported by the human subjects. Should a particular property, or range of values of a given property, prove to be of particular interest (perhaps because of a large variation in subjective value for a small change in objective value) it is rarely practical to repeat the tests with a greater definition, as the same human subjects may not be available or may give less consistent responses (for health or other reasons) when the test is repeated, perhaps several weeks after the original test. Any complex system comprises a large number of network elements, each having a variety of possible behaviours, which will interact with each other. It would not be practicable to perform subjective tests using human subjects for more than a small proportion of the possible network behaviours. Furthermore, in a packet switching system each individual data packet may be transmitted by a different route, using different network elements. Since degradation will in general be different for each such route, the effect on the complete message will be much more difficult to assess.
The present Applicant's Internationl Patent specifications WO96/06495 and WO96/06496 (both published on 29 Feb. 1996), and WO97/05730 (published 13 Feb. 1997) propose three other possible solutions to this problem. WO96/06495 describes the analysis of certain characteristics of speech which are talker-independent in order to determine how the signal has been modified by the telecommunications link. In WO96/06496, the content of a received signal is analysed by a speech recogniser and the results of this analysis are processed by a speech synthesiser to regenerate a speech signal having no distortions. WO97/05730 discloses a process in which the received signal is compared with parameters identifying speech-like and non-speech like characteristics, to derive a measurement of the quality of the signal. However, all of these techniques require considerable processing power, making it difficult to monitor signal quality in real time, and the processes all require some assumptions to be made about the incoming signal.
A further proposal, disclosed in the Applicant company's International Patent Application PCT/GB01/02050 filed on May 10, 2001 provides a method using the simpler “intrusive” measurement techniques in a non-intrusive manner on a communications channel which is carrying a live call, intended for use for voice communication systems, in which there are normally gaps in the speech content. A predetermined test signal is transmitted over the communications link during such gaps, and compared at the remote end with a reference copy of the signal. However, this system is only suitable for systems in which there are suitable gaps in the information content. It is not suitable for systems in which such gaps are used for other purposes, such as carrying data or other voice calls, or for “half-duplex” systems in which both directions of conversation use the same channel.